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METHOD AND APPARATUS FOR PRODUCING A COORDINATED GROUP 

ANIMATION BY MEANS OP OPTIMUM STATE FEEDBACK, 

AND ENTERTAINMENT APPARATUS USING THE SAME 

BACKGROUND OF THE INVENTION 
i- Field of tlxe Invention 

The present invention relates to a method and 
apparatus for producing an animation to represent in a 
quasi fashion the movement of a group including a 
10 plui|ality of members such as persons or animals, and 
mor€ particularly, to a method and apparatus for 
controlling the movements of members as a group. The 
present Invention also relates to an entertainment 
appezratus using such a method. 
15 2, riescrlption of the Related Art 

Conventional computer graphic techniques can be 
classified into two categories: group animation and 
physically based animation. 

Physically based animation refers to an animation 
20 which is produced in accordance with a physical 
principle such as a law of mechanics. More 
specifically, equations of motion (simultaneous 
nonJinear equations or differential equations) is 
determined according to Newton's law or Lagrange's law 
25 for each member. However, the cost of precisely 
simulating mechanical or physical motion is high. 
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Th©i}efore, in the art of information processing 
apparatuses « such as computers, or In the art o£ 
entertainment apparatuses^ to which the present 
invention is applied, some simplification is generally 
ormed to reduce the cost. 

That is I in these fields of technologies, it is 
necessary to build a physically accurate model, 
equations of motion based on a quasi-physical 
modell may be employed if the equations allow the 
10 motion of the respective members of interest to be 
determined in real time and if the resultant motion 
appears sufficiently natural. 

(1) Group Animation to Represent Group Behavior 
15 The research work by Reynolds has had a profound 

impact upon the art (C. W. Reynolds, ''Flocks, Herds, 
and Schools: A Distributed Behavioral Model", 
Prodeedings of Siggraph 87, Jul., 1987, 25-34). After 
his research work, animation of a group of birds, 
20 fishes, or animals can be realized by means of 

programming independent members of the group. By 
adjusting parameters associated with the behavior of 
eacli member, it is possible to control the behavior in 
a discrete fashion. However, when the group is viewed 
25 as a whole, most of the group behavior occurs 
spontaneously. 
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Tu and Terzopoulos have produced a more realistic 
anin^atlon of a group Including a plurality Of members 
by llmproving the reality of the group and the senses 
by ^imulating blomechanical behavior (X- Tu and D, 
Terzjopoulos , "Artificial Fishes: Physics, Locomotion, 
Perception, and Behavior", Proceedings of Siggraph 94, 
Jul-r 1994, 24-29). This pioneering research work 
indljcated that the behavior of a group of fishes or 
anin^als can be synthesized m a more natural fashion 
by Employing a combination of stimulus -response rules. 

Receiving a stimulus from artificial intelligence 
tech|nology, Funge, Tu, and Terzopoulos have proposed 
and demonstrated a very elegant framework which allows 
the irepreeentation of perceptive behaviors Of members 
of a group (J- Funge, X. Tu, and D. Terzopoulos, 
"Cognitive Modeling: Knowledge, Reasoning and Planning 



for 



Intelligent Characters", Proceedings of Siggraph 



99, jJul- 1994, 29-38). They have attained their 
objejctive in that artificial life and the behavior 
20 thedeof are animated. 



25 



(2) Dynamic and Physical Group Animation 

j Dynamic and physical animation has great problems 

in eitability and controllability. The problems become 

j 

seriious when groups include a large number of members. 



The 



motion of group members is very complicated, and 
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it has toeen thought that it is very difficult to 

I 
I 

matliematically represent the complicated motion. 

} 
t 

Howejver, two physicists, Toner and Tu, have recently 
expljained using a mathematical model how coordinated 
5 groiip behavior occurs spontaneously (J. Toner and Y, 
Tu, "Flocks, herds, and schools: A Quantitative Theory 
of blocking" , -Physical Review Letters, Vol- 58, 1999). 
A pxtofound understanding of the motion of a group 
including a plurality of members is important. In 

10 partjicular. in the art of computer animation, the 

1 
I 

deveilopment a high-efficiency method of dynamically 

i 

conljrolling a group including a plurality of members 
is zjequired. 

IBrogan and Hodgins have investigated the behavior 
. 15 of ai group with significant dynamics and have 



presjented their opinion about the complexity of the 

protjlem (D, Brogan and J, Hodgins, '•Group Behaviors 

I 

f or jsystems with Significant Dynamics"^ Proceedings of 
the jl995 lEEE/RSJ International Conference on 

20 Intelligent Robots and Systems, Vol. 4, 528-534; D. 
Brogan and J. Hodgins, "Group Behaviors for Systems 
witlt Significant Dynamics"^ Autonomous Robots 4(1}, 
1997|, pp. 137-153). That is, they have pointed out 
that! motion of individual members and local 

25 interactions make it difficult to control the group as 
a wbiole and that a combination of local knowledge and 
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glottal knowledge is required to efficiently control 
the jindividual members . 

iThe objective of the research work by Brogan.and 
Hodgiins was not to provide a technique for animation 
5 but :to provide a technique which may be applied to the 
art jof robotics in which realistic equations of motion 
can jbe employed so that a realistic group behavior ia 
reprjesented by means of dynamic control. Therefore, 
this! teohniqu© is not assumed to be applied to 
10 tecKnical fields such as animation in which it is 
req\:jired to simplify the motion of the individual 

I 
I 

memders. depending upon the specific purpose. 

1 Furthermore, in the conventional techniques 
described above, control factors are adjusted 
15 expejrimentally and manually, and automatic adjustment 
of tjhe control factors has not yet been achieved. 



p SUMMARY OF THE INVENTION 



•In view of the above, it is an object of the 
20 presient Invention to provide a method and apparatus 
for iproducing an animation of a group including a 
plurlality of members whose behavior is dynamically 
contjrolled. It is another object of the present 

inveintion to provide an entertainment apparatus which 

I 

25 uses! an animation produced by the above method. 

j According to an aspect of the present invention. 
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to Achieve the above objects, there is provided a 
method of producing an animation of a group including 
a plurality of members, the method comprising a 
continuous processing step for determining the states 
5 (such as a location and a dynamic state) of the 
respective members at each time step in accordance 
witli predetermined dynamic characteristics of the 
respective, members, and a discrete processing step for 
accepting, from the outside, a command specifying the 

10 ovei|all state (such as a shape or a dynamic state of 
the igroup) to be achieved for the group and assigning 
roleis to the respective members of the group in 
acoclrdance with the overall state specified by the 
comniand. In the method, if new roles are assigned to 

15 members in the discrete processing stepr the dynamic 
states of the respective members are adjusted, in the 
continuous processing step, in accordance with the 
assigned new roles, 

; According to another aspect of the present 

20 invention, there is provided an entertainment 

apparatus comprising an input unit for inputting a 
comniand issued by a user or an operation performed by 
a u^er, an executing unit for executing a prestored 
entertainment program in accordance with the command 

25 or Operation, and an output unit for output ting the 
resiilt of th6 exeoution of the entertainment program. 
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In the apparatus the executing unit produces an 
anirftation of a group including a plurality of members 
usirig the above-described method according to the 
present invention, and outputs the produced animation 
5 to the output unit so as to display the animation. 

BRIEF DESCRIPTION OP THE DRAWINGS 
I Fig. 1 is a schematic diagram illustrating an 
example of an animation produced using a method of 
10 controlling the movement of members as a group 
according to the present invention; 



ru. 



:Fig» 2 is a block diagram illustrating an 
embodiment of a system for performing optimum hybrid 
% feedback control according to the present invention? 

F 15 \ Fig. 3 is a schematic diagram illustrating another 

=^ ■ i 

m example of an animation produced using a method of 



controlling the movement of members as a group 
according to the present invention; 

: Fig. 4 is a schematic diagram illustrating an 
20 example of role replacements according to the present 
invention « wherein changes in the trajectories in the 
state space are shown for a group including three 
members ; 

;Fig. 5 is a schematic diagram illustrating an 
25 example of role replacements according to the present 
invention; 
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Pig. 6 is a diagram illustrating an example of a 
run-itime coordination algorithm according to the 
present invention; 

:Fig. 7A is a graph illustrating the performance 
5 index of a system according to the present invention 
at a function of the frame number « which corresponds 
to the elapsed time, and Fig. 7B is a graph 
illustrating the number of local interactions in the 
syst^em according to the present invention as a 
10 function of the frame number, which corresponds to the 
elapsed time; 

^ IFig* 8 is a diagram illustrating an example of an 

?. 'ij 

algcjrithm of a version optimized in terms of a block 
diagonal system; 



it fit? 



as 

. 15 Fig. 9 is a table illustrating the evaluation 



results in terms of the stability and the calculation 
load during a game, for the algorithm according to the 
present invention; 

•Fig, 10 is a schematic diagram illustrating one 
20 scene of an animation of a group of bees produced 
according to the present invention; 

[Fig. 11 illustrates examples of matrices used to 
generate the animation shown in Fig. 10; 

Fig- 12A is a graph illustrating the evaluation 
25 funqtion (optimal cost) for another example of an 

animation as a function of the frame number, and Fig. 



8 



12B is a timing chart illustrating the changes in the 
state space in terms o£ the roles assigned to six 
airplanes contained in an formation of airplanes used 
in ih& example shown in Pig. 12A; 

;Fig. 13A is a graph illustrating the performance 
index associated with another example of an animation 
according to the present invention as a function of 
the frame number, and Pig. 13B is a timing chart 
illustrating the changes in the state space in terms 
of the roles assigned to six fishes contained in a 
group used in the example shown in Fig. 13A; 

Fig. 14 is a schematic diagram illustrating an 
exartple of an off-line algorithm for calculating 
parameters associated with a coordination process 
according to the present invention; 

i Pig. 15 is a schematic diagram illustrating 
another example of an off-line algorithm for 
calculating parameters associated with a coordination 
process according to the present invention; 

: Fig. 16 is a functional block diagram of an 
entertainment apparatus using a control method 
according to the present invention; 

; Fig. 17 is a schematic diagram illustrating an 
example of the hardware configuration of an 
entertainment apparatus using the control method 
according to the present invention; and 
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Fig- 18 is a circuit diagram illustrating an 
ex£iinpld of a circuit configuration of the 
entertainment apparatus using the control method 
according to the present invention. 

5 

DESCRIPTION OP THE PREFERRED EMBODIMENTS 

With reference to the accompanying drawings, 
embodiments of a method and apparatus for producing an 
animation of a group including a plurality of members 
10 whose motion is coordinated by means of dynamic 

control according to the present invention, so that 
the members behave in a desired fashion as a group, 
and also an embodiment of an entertainment apparatus 
using the method are described. 



:?sss 

1^ . 15 



(A) Method and Apparatus for Producing Group Animation 
: A typical example of cooperative group behavior is 

the : systematic or coordinated behavior of a group of 

fishes or animals. However, the group behavior 
20 appearing at the surface is not always controlled 

perfectly I ' as is the case in a group falling down from 

the Itop of a cliff at full speed. 

In the natural world, when there is no control 

system which controls the entire group, the relative 
25 motion of each member of the group is tightly 

constrained by local interactions. In many cases. 
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local interactions occur not as an objective action 
but as a by-product. 

In the present embodiment, by way of example « a 
group of fishes or animals is animated by means of a 
5 dynamic simulation and on the basis of a physical 
model. Herein it is assumed that the intrinsic 
characteristics of the group are poorly controlled and 
thus the group behaves as a potentially unstable 
dynamic system. In the present embodiment, taking into 
10 account the above, the physical characteristics 

associated with the motion of the members of the group 
are reflected in the animation, 

(A-1) Motivation for Use of Dynamic Group Motion 
15 In the present invention, the term "dynamics" is 

used to describe a general dynamic system. This is 
applied not only to a physical model or a quasi<- 
physical model but also to a model which describes the 
behavior by mathematical expressions such as a 
20 combination of differential equations and finite 
difference equations including a discrete event 
component in a finite state machine or the like. 

; In computer animation, the application of dynamic 
characteristics eliminates a lot of tedious manual 
25 tasks required in synthesis of the motion of a group 
including a very large number of members- If the 
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motion of a group including a plurality of members is 
reptesented on the basis of a model rather than 
trajectories, it becomes possible to easily make 
modifications to adapt to various requirements at the 
5 design stage. That is, even if the environment or the 
scenario is modified many times, the use of a model 
allows trajectories to be easily calculated for the 
modified environment or scenario* 

In interactive animations and video games in which 

10 an Infinite number of events can occur in response to 
unpredictable operations performed by a user, 
employment of the group animation based on the dynamic 
characteristics is very useful. In order that the 
motion of a group is controlled by a scenario or a 

15 game player without falling into a confused and 
uncooperative state, but intentional or forced 
coordination, rather than spontaneous coordination, 
among the group members is required. 

20 (A- 2) Main Improvements 

If animation of a group including a plurality of 
members on the basis of independent behaviors of the 
individual members is attempted « it is difficult to 
control the group in accordance with a given scenario. 

25 In order to form a. group in a desired fashion with a 
required precision at a correct time, it is effective 
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to give a command to control the group as a whole with 
coordination. To this end, the following improvements 
are required. 

First, high-precision coordination among the 
5 individual trajectories ie required to* realize 

systematic behavior. Essentially, this is a problem of 
continuous control. To the above end, it is desirable 
to link, to a continuous world, a program or an 
inference engine operating according to rules based on 

10 the discrote behaviors and empirically obtained 
knowledge, more tightly than is currently linked. 
Second, problems such as instability and 
uncontrollability intrinsic to dynamic systems often 
occur in physical animations. In the present 

15 embodiment, in view of the above, the following 
improvements are made. 

1, Unified Rule for Obtaining Coordinated Motion of a 
Group 

20 The concept of a state- space layout is introduced 

herein to provide a unified fundamental expression to 
specify the motion or the state of a group as a whole. 
Furthermore, it is proposed herein that when a dynamic 
state becomes diverged, a hybrid rule consisting of 

26 continuous and discrete processes is employed to 

efficiently control the dynamic state into a stable 
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State . 

2. Coordination Algorithm at Run Time 

We herein propose a coordination algorithm for 
controlling individual members of the group frame by 
frame instead of treating them as a mass* This 
algorithm exerts high performance in real time when 
being applied to a small group including a small 
number of members. 

3. Guideline for Adjustment of Dynamic Model 

. The optimum performance of this coordination 
algorithm is assured for linear dynamic 
characteristics • For complicated dynamic 
charaoteristics , a guideline is presented for 
adjustment of the dynamic model to handle nonlinearity 
and maintain high performance « 

4 • -Demonstration 

Fig. 1 illustrates an example of an animation 
which has a high dynamic level and thus which could 
not be achieved by any other method because of the 
problems of instability and uncontrollability. In Pig. 
1 , the result of the pioneering work by Tu and 
Ter2opoulo$ (X. Tu and D. Terzopoulos, "Artificial 
Fishes: Physics, Locomotion, Perception, and Behavior", 
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Proceedings of Siggraph 94. Jul., 1994, 24-29) is 
herein reproduced by the technique according to the 
present invention. Herein, coordinated motion of a 
group is realized in an animation such that the group 
5 behaves systematically with high- accuracy dynamic 
characteristics. Thus, the group is maintained in a 
coordinated form without falling into a diverged state 
regardless of a disturbance. In the example shown in 
Pig. 1, a group of fishes having physical 
10 characteristics moves against the flow of water, and 
the group of fishes is illuminated with light. 

(A-3) Problems to be Solved 

In the present embodiment, the group coordination 

15 is separated as an intermediate layer between dynamic 
members and the interactive scenario. In the 
coordination according to the present embodiment, 
roles and control commands are issued to the 
respective members in response to a given group-level 

20 command so as to control the members to form a group 
in a desired fashion. 

: A first assumption made in the present embodiment 
is that a command specifying a group -level state can 
be issued as a layout command wherein the layout 

25 refers to a combination of operating points in a state 
spape. Before describing the concept of the layout, it 
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is first described that a command to control members 
to form a group can be described by one of the methods 
listed below or one of various other methods: 

(a) A combination of labeled or non-labeled target 
locations described globally, locally, or on a frame- 
by-frame basis; 

(b) A verbatim description of the manner of 
forming a group; 

(c) Redistribution map of the member density in a 
particular scene; and 

(d) Specifying a dynamic level by a velocity 

vector. 

In order to illustrate the effectiveness of the 
above methods, the manner in which the group changes 
in the state space with time is described below. 
Herein, the term "group" is used to describe a set of 
members whose motion variables are gathered into 
corresponding state vectors. 

A member state vector with a height (vector 
size) of n. includes all variables at a time step n, 
wherein the variables include a variable representing 
theidegree o£ freedom of a member i. first derivatives 
which represent the dynamic state thereof, variables 
representing physical and quasi -physical states, and 
the internal state of motion. Member state vectors are 
combined together into a single state vector X„ « [x„\ 
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x„*,..., x^"]' with a height of N-He to represent the 
state of the entire group. In this teohnigue, the 
layout is not directly linked either to the respective 
individual members or the target layout, but the 
5 layout represents the distribution of non-labeled 
operating points for providing blind assignment to a 
group as a whole. That is, the layout is given by a 
vector X„« - [x,.,S x,,,',..., st,,*"]' (200 in Fig. 2). 

To realize the target layout, roles of members are 

10 assigned by a role assignment switchboard (201 In Fig. 
2) by rearranging the target and assigning roles to 
the individual members. Herein, this is represented by 
n;„-Xt,£ where the N-n, x N-n^ matrix represents a role 
assignment at time n, that is, a bisection between the 

15 target and the members (R. Grimaldi, "Discrete and 
combinatorial mathematics", Addison-Wasley, New York, 
1999) . 

Such a matrix expression is very convenient and 
will be used in the remaining part of the description 

20 based upon linear algebra. However, in some oases, a 
simple mapping table can function well. Therefore, in 
practical systems, the matrix expression is not 
necessarily required. For example, the target - 
assignment for members 1, 2, 3, 4, and 5 (selection of 

25 possible 51 = 120 bisections) can be described as 
follows : 
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Furthermore, in the present embodiment, the 
optimum hybrid feedback problem 1& solved as follows: 

(1) When a fiet of N members is given, rhe dynamic 
characteristics of the respective members and the 
conditioned responses linked to one another are 
collectively represented by a nonlinear mapping f as 
follows: 



15 



X^., = f (X„ UJ 
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(ii) k layout is defined for the group as a target 
state in the state space X„£ in accordance with a 
command issued by a user or in accordance with a game 
scenario. That is, the layout is specified on-the-fly. 

(iii) A coordinated control = g(X„-n:».'X„j) is 
determined which allows formation of a group directed 
to the layout with the optimal performance (tradeoff 
between the time response and the energy consumed by 
the members). This reduces to a hybrid ( continuous - 
discrete) problem described below: 

(iii-a) Continuous Part: X„ is driven toward n^-'^ret 
using the control U„ while minimizing a trade-off 
function J between particular energy and time. Instead 
of the function J representing the integral coat, an 
evaluation function in a quadratic form in the optimal 
regulator (LQ) problem may also be employed. 

(iii-b) Discrete Event Part: Replacement of roles 
assigned to members is performed if it is determined 
that role replacement of results in an improvement 
in performance (for example, a reduction in the cost 
function J). More specifically, the cost J required to 
maintain the present roles of the individual members 
and the cost J* required to replace the roles among K 
members having the lowest performance, which will be 
described later, are calculated and compared with each 
other to determine whether the replacement of roles 
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results in an improvement in performance. In the case 
where the total number N of members is small, K may be 
selected to be equal to N. However, when N is large, 
replacement of roles is performed only among K members, 
where K < N. 

In the above-described optimum hybrid feedback 
technique according to the present invention, the 
problem of the group coordination required to achieve 
group --level control is intentionally separated from a 
large number of other problems in the group animation. 
On the assumption that dynamic effects are dominant, 
the dynamic effects are formulated on the basis of the 
viewpoint of the dynamic system and the control 
thereof, and individual members are separated from a 
high-level layer (planning and recognition) widely 
used in multiagent or cooperative robotics on the 
basis of lA driving. However, note that the separation 
does not mean inhibition of use of the layer related 
to the system. 

Fig. 2 illustrates, in block diagram form, an 
example of a system for realizing the optimum hybrid 
feedback control according to the present invention. 
This system includes the role assignment switchboard 
201, a hybrid coordination processor 202, and a member 
state determinator 204. 

In accordance with a layout 200 specifying a 
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particular group-level state which has been set in 
accordance with a request given from the outside, the 
role assignment switchboard 201 assigns roles required 
to achieve the layout 200 to the respective members. 
5 The hybrid coordination processor 202 includes an 

optimum continuous controlling unit 202a and an 
optimum discrete role assigning unit 202b. The optimum 
continuous controlling unit 202a performs continuous 
feedback control, in accordance with the current group 
10 state, in the reference state in which the roles have 
13 been assigned by the role assignment switchboard 202 

ffl to the respective members^ thereby continuously 



w 



determining the states of the respective members. The 
optimum discrete role assigning unit 202b controls the 

15 role assignment switchboard 201 so that the roles of 
the members are reassigned at predetermined times, in 
response to an external command, or depending upon the 
present state (snapshot) of the group, 

A member state determinator 204 determines the 

20 states of the respective members from a set of 

variables Ei describing the dynamic characteristics 
based on the states of the respective members 
determined by the optimum continuous controlling unit 
202a, taking into account a global disturbance and 

25 interactions among the members, thereby determining 
the state of the group at the current time. 
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In the hybrid coordination process performed by 
the system shown In Fig. 2, taking into account the 
internal dynamic characteristics of members by 
introducing the concept of encapsulated members is 
5 important in designing of the feedback control. The 
coordination acts as a clutch between the interactive 
scenario and members, and the group is treated not as 
a black box but as a white box {model having a known 
structure) . 

10 The hybrid coordination process shown in Fig- 2 

Q makes it possible to realize an interface between 

IQ discrete planning and continuous dynamic 

IH characteristics via coordination between discrete 

''H motion and continuous motion. 

.15 In many cases, the hybrid coordination process 

according to the present invention can be 
advantageously used. Fig. 3 illustrates one example. 
P In the example shown in Fig. 3, a large number of 

airplanes fly as a group, in which pilots are required 
20 to fly their airplanes in coordination with each other. 
In this case, because the continuous feedback control 
becomes more dominant than the discrete determinations, 
it is difficult to represent the flights on the basis 
of conventional synthesis rules , The coordinated 
. 25 flight animation shown in Fig. 3 will be described in 
further detail later. 
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(A- 4) Theoretical Background 

The present Invention is based upon the state 
space approach (disclosed in M. van de Panne, E. Fiume, 
and Z. G. Vranesic, "Reusable Motion Synthesis Using 
State-Space Controllers", in Proceedings of SIGGRAPH 
1990, Computer Graphics Proceedings, ACM SiGGRAPH, 
1990, 225-234) and also based upon the response 
feedback control. 

Furthermore, a plurality of members are 
simultaneously controlled within a linear optimal 
control freunework (disclosed in L. Brotman and A. 
Netravali, "Motion interpolation by optimum control", 
in Proceedings of Siggraph 88, Aug. 1988, 309-315) 
wherein the control of the plurality of members 
includes replacement of roles assigned to the 
respective members . 

Prom a theoretical viewpoint, the method according 
to the present invention is characterized by the 
hybrid control processing consisting of time -sampled 
continuous processing and discrete event processing, 
which are combined together and related to each other. 

(A-4-1) Hybrid System and Hybrid Control 

More particularly, this system is a so-called 
switched system including a group of continuous time 
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subsystems which are set in correspondenae with 
specific role assignments and also including a rule of 
coordinating the switching (role replacement) among 
the subsystems. The field of such a hybrid system is 
interdisciplinary and is at the boundary between 
control technology and computer science. Applications 
o£ the hybrid system are expanding into other fields 
such as automobile, electrical power supply, aircraft, 
and traffic control technologies (Special issue on 
Hybrid Control Systems, IEEE Control Systems Magazine, 
19(4), August 1999) in which themes of animations of 
interest include a collision, traffic, etc. 

(A-4-2) Coordination of State-Space Trajectory 

■ Controlling a group including a plurality of 
members such that the group has a desired form can be 
regarded as a problem of feedback control. The dynamic 
characteristics of a group involve not only 
conditioned responses and interactions (for exa.mple, 
instinctive behaviors of a group) but also many other 
events related to one another . (for example, 
controlling of machines, muscles, and motors). Such a 
system is generally represented by a nonlinear mapping 
f with a combination of state variables. In the 
present embodiment, the group as a whole is regarded 
as a dynamic system and formulated. The formulation is 
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then applied to the group animation. 

In the present embodiment, the formulation of the 
coordinated motion of the group reduces to finding an 
admissible control set {U„,...Uj which minimizes the 

performance index J({U^o ^J' CX^«...Xj) constrained 

by a particular evaluation condition or by Equation 
(3) descried below. 

• Vne[n,. . ."I: X„,i - f(X.. UJ 
...(3) 

Although infinity is employed in Equation 3, the 
actual settling time (the time required for the state 
variables to settle within ±5* around the target 
values) can be generally reduced to an arbitrary value 
by increasing the control level. In this nonlinear 
optimization problem, the processing of trajectories 
and inputting of tasks are required over the period of 

time [1 • • • 

In applications to interactive animations, global 

searching is impossible and thus it is necessary to 

perform a process at each time step. For the above 

reason, a closed control rule U„ = g(X„) is required to 

convert the optimization problem into an optimum 

feedbaclc problem, wherein g is a nonlinear mapping. 

It will be useful to discuss here the treatment of 



25 



SC1626 

nonlinearity based on linear design before discussing 
the nonlinear control (J. Slotine and W. Li, "Applied 
Nonlinear Control". Prentice Hall, 1991, 461 pages). 
This approach is well known to those skilled in the 
5 art, and it has been established on the basis of a 
reliable theory. For example, gain scheduling taking 
into account groups to be switched based on a linear 
model is described in D. A. Lawrence and W. J. Rugh, 
"Gain Scheduling Linear Dynamic Controllers for a 

10 Nonlinear Plant". (Automatic. 31(3), 1995. pp. 381-390. 
and robust control in which nonlinearity is reduced to 
a disturbance is described in K. Zhou, "Essentials of 
Robust Control", (Prentice Hall International, 1998. 
The above approach is a practical choice in the 

15 art of animation technology. That is, in virtual 

worlds, it is reasonable to manually build a dynamic 
model such that it behaves well. It is interesting to 
note that two physicists. Toner and Tu, who have 
proposed the mathematical theory of producing a group, 

20 have both employed similar two-step approaches in 

which linear approximation is performed first and then 
nonlinearity is taken into account (J. Toner and Y. Tu, 
"Flocks, herds, and schools: A Quantitative Theory of 
Flocking". Physical Review Letters, Vol. 58, 1999). 

25 The extraction of nonlinearity and the separate 

treatment thereof bring the problem back to the field 
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of linear feedback control theory* Thus, it is assumed 
herein that the dynamic characteristics of a natural 
system can be represented by a set of matrices A and B 
as described below, 

X,., = AX, + BU„ 
...(4) 

vhere the matrix A can be represented in the following 
form (the matrix B can also be representing in a 
similar form) s 

p.] ■ 1 

0 0 

A - [e,] 

0 0 

1 4 4 442 4 4 4 43 
Nn^ 

Square matrices of order of ne appearing on the 
diagonal line of the matrix A represent the dynamic 
characteristics of the respective members. On the 
other hand, elements C^^ lying off the diagonal line 
may be used to represent "natural" couplings among the 
members (due to, for example, the instinct of the 
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group). Such a conditioned response is hard-wired and 
embedded at a low level in the dynamic characteristics. 

In another embodiment, elements of the matrix B 
may be used to identify a command which is issued to a 
5 single member but which can influence a plurality of 
members and to share the same motivation among 
subgroups * 

The fundamentals of linear optimization control 
and linear quadratic (LQ) synthesis (G, F. Franklin, J. 

10 D. Powell, and M. Workman, "Digital Control of Dynamic 
Systems", Addison Wasley, 1996) are discussed below. 
The multivariable optimization control rule g can be 
given by a matrix gain G which minimizes a performance 
index (which will be described below) represented by 

15 the state-space trajectory of the feedback system: 



Because J has a quadratic form, a local 
20 minimization condition is required. This condition is 
sufficient to achieve a global minimum value. This 
evaluation criterion has a fixed structure and 
provides to a designer a means for adjusting the 
dynamic behavior by matrices Q and R while making a 
25 compromise between the response time of a selected 
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state variable and the performance level (energy) of a 
process. 

In the linear oase, Riooati^s equation (RE) is 
obtained by solving the minimization problem under 
5 constrained conditions by means of Lagrange's method 
of undetermined multipliers: 

Sn * A'[S„.i-S,,,B(R+B*S,,,B)-^B'S„,jA+Q -..(7) 

10 In order to solve Riccati's equation, it is 

required to apply Equation (7) starting from the end 
point in a reverse direction along the time axis so as 
to obtain a time-varying control rule (G» P. Franklin, 
J. D. Powell, and M. Worman, "Digital Control of 
^ 15 Dynamic Systems", Addison Wesley, 1988). Fortunately, 
it is known that the time-varying control rule quickly 
becomes stable, and the time-varying control rule can 
be well approximated by increasing n toward co. Thus, 
we can obtain a steady state version of Riccati's 
20 equation (ARE) to be considered, as described below: 

S. = AMs«-S,B(R+B'S.B)"'bX1A+Q ...(8) 

It is generally impossible to obtain a solution in 
25 a closed form for Equation (8) except for very simple 
cases. Thus, a numerical approach is employed at this 
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stage- In Section (A-8), the eigenvalue expansion 
procedure which is widely employed in various CAD 
packages used in the design of control systems will be 
briefly described. If S« is determined, the optimum 
5 feedback rule is given as follows: 

...(9) 

10 The control gain G is a perfect matrix, that is, a 

multivariable controller which performs coordination 
^0 taking into account coupling among the members and 

which minimizes the evaluation criterion at the group 
'f; level. In contrast, a diagonal matrix or a block 

sip 

U 15 diagonal matrix controls . the respective members 
p independently. 

Q (A-- 4 -.3) Coordinated Response to Discrete Events 

In the above discussion ^ it is assumed that the 
20 initially assigned roles are valid, that is, n, Jt^^aiCe 
over a period in which a particular layout is given. 
However « such a stable state is not realistic in time- 
varying environments which often occur in interactive 
games. In order to achieve effective coordination in a 
25 group, it is required to replace roles between members 
to compensate for various events and factors, whenever 
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there is an opportunity to perform replacement. Tlie 
opportunity for role replacement can occur when one of 
the conditions described below is met« Note that/ the 
role replacement opportunity is not limited to those 
described below; 

(a) Reduction in run- time performance due to a 
conflict between the nonlinear dynamics and the linear 
assumption; 

(b) Addition or removal of a member to or from a 
group r which causes necessity for on-the-fly 
rearrangement of a layout; and 

(c) Occurrence of a discrete event such as a usual 
collision or non-modeled uncertainty in a time -varying 
environment . 

In addition to the opportunities described above, 
replacement of roles may also be performed when a 
command is issued by a user or by a scenario to 
generate a new group ^ or when a change in the number 
of members occurs, or otherwise when the motion of 
members is affected by an obstruction or a collision. 

In practical application to an animation, it is 
required to determine the states or locations of the 
respective members frame by frame « However, the 
assigned roles may be maintained over a plurality of 
frames, and replacement of roles may be performed only 
when the degree of necessity for the replacement 
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becomes v©ry high. In this case, it is required to 
strictly define the conditions which should be met to 
perform the role replacement. Furthermore, in this 
case, the evaluation or comparison of candidates to 
5 determine K members to be replaced in roles is 

inhibited or K is set to a small value over a period 
between events which can trigger the role replacement. 

The effects of replacement of roles are described 
below with reference to Fig. 4. Fig. 4 illustrates 
10 trajectories in the state space for a group including 
three members. In Fig. when an event occurs in an 
initial state space 401 in which a target layout 410 
has been set in advance, if replacement of roles is 
performed, the state space 401 is changed to a state 
15 space 403* However, if replacement of roles is not 
performed, the initial state space 401 becomes equal 
to the state space 402. 

The assignment of roles is performed in accordance 
with a state snapshot. This becomes possible because 
20 of a significantly important theorem of optimum linear 
control theory which is described below. That is, the 
cost required to drive a dynamic system into a target 
state n*X^^f can be estimated as follows: 

25 J = [X„-«,'X^eel'SjX„-7C„'X,,,] 

...(10) 



as 



'.•tB;:. 
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The role replacement, that is, alternative 
bijections (k>/lc « 1...N! between members and the 
target can be evaluated and compared with one another 
5 by numerically evaluating the evaluation gains 

obtained by the respective options thereby evaluating 
the degrees of acceleration of the convergence toward 
the specified target achieved by the respective 
options. This is illustrated in further detail below 
10 with reference to Fig. 5. 
0 In the example shown in Fig- 5, the members are 

ranked in accordance with the ease of reaching the 
targets of the members by estimating the Integral cost 

15 approximated by ^ " ^* If S. is 

M replaced with an identity matrix, the result decreases 



Q 



to a Euclidean distance. However, the reduction in 
distance does not always result in an increase in the 
degree of ease of reaching targets. For example / 
20 although fishes 4 and 5 are near the targets, they 
have to change their direction of motion into a 
lateral direction to reach the targets. Note that 

updating of lis performed in a rather shortsighted 
fashion and the best option is immediately employed. 



25 



(A-S) Algorithm 
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An example of a novel algorithm for achieving 
coordination at run time according to the present 
invention is shown in Fig. 6. This example of the 
algorithm ehown in Fig. 6 has a high responsiveness 
and may be performed in real time. Furthermore, at the 
design stage, the optimum feedback rule is determined 
from the characteristics of the dynamic system 
according to the evaluation criteria using an off-line 
algorithm including a procedure widely used to solve 
the LQ problem. This will be discussed in further 
detail later in Section {A-8). 

The algorithm shown in Pig. 6 is applied for each 
time step. The calculation efficiency of the algorithm 
may be increased by various methods including a 
clustering technique used in a simulation of a many- 
body (n-body) problem or a method adapted to 
characteristics intrinsic to a particular case. 
Although a matrix representation is employed, the 
matrix representation lis not always efficient. 

Because the complexity of the role assignment 
increases as the factorial of the number of members / 
the above algorithm cannot be scaled up unless the 
role replacement is limited to a small subgroup. 

In essence, K members having the lowest 
performance (worst performers) and thus having a large 
influence upon the overall performance index J are 
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tracked so that when an opportunity for role 
replacement is found, role replacement Is performed 
among those worst performers, thereby reducing the 
calculation load from N! to Kl (K < N or K << N) . When 
the calculation is performed in real time, K generally 
has a small value such as 6. For example. K and N may 
be set such that K =7 and N = 100. When N is small, an 
initial sorting process is skipped, and thus a 
corresponding overhead is eliminated. 

(A- 5-1) Modification Associated with Independent 
Member Dynamics 

When coupling among members is negligible, 
separate optimum controllers (which may be obtained by 
means of LQ synthesis) are used for the respective 
members (j). The contribution to the overall 
performance is given by J(j) = - (Ji»'X«£)^rs*""''lXft^ 
- («„'X„j)»] where (itR-X,.,)* represents the 3th coordinate 
of the current reference vector 3t„'X„j. Note that the 
role assignment and the layout are maintained at ;i„ and 
X„£, respectively. By comparing not the absolute cost 
but. the derivative thereof and employing factorization 
in the calculation, the complexity of J is reduced 
from 0(N*) to 0(N) . 

(A-5-2) Convergence Analysis 
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The control theory incluaes a tool for evaluating 
the dynamic behaviors. This tool is useful to check 
whether the specified layout is maintained. If the 
dynamic characteristics based on which the control is 
5 performed are linear, it is possible to obtain formal 
proof of convergence from the essential stability of 
the LQ control (F. Franklin, J.D. Powell, and M. 
Workman, "Digital Control of Dynamic Systems", Addison 
Wesley, 1998). According to Liapunov's method (J. La 
10 Salle, J., and S- Leffscheta. "Stability by Liapunov's 
Direct Method", Academic Press, New York, 1961), the 
system is globally stable when the Liapunov function 
satisfies the following conditions: 

1. V(Xo) is continuous and positive. 
15 2. V(X„) decreases with time. 

3. There is a function of (X„) which satisfies the 
following conditions: 

(a) 0 < (I |XJ I) < V(XJ 

(b) lim ( I !xj I )=« 00 when | |xj | « 

20 Because the stability of the LQ problem is ensured 

(G. F. Franklin, J. D. Powell, and M, Workman, 
"Digital Control of Dynamic Systems", Addison Wesley, 
1998), a Liapunov function V(X„) « X^'SJJo which 
satisfies the conditions I and 3 described above is 

25 selected. 
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AV = V(X,,J-V(XJ 
...(11) 

Herein, S. = A'[S. -S„,,B(R+b'S„,iB)-'BX,i1A + Q, and 
thus V = -X„*[Q + G'RG]X„. Herein, IQ + G*RG1 is 
positive and finite, hence V < 0. This ensures the 
stability. That is, when assigned roles are fixed, 
convergence toward the designated layout is assured. 
In the present case, the replacement of roles 
{permutation «„) provides an opportunity for a further 
reduction in the calculation load, and thus the 
convergence is accelerated. 

. V(rt,X„.i)<V(X„,J V(3i„X.,J-V(X„.i)<0 
. . .(12) 

Equation 12 indicates that the gain can be further 
increased by optimizing the time sequence {jto, . . . , n^, 
31b+i, ....» ^ associated with the assigned roles. 
Herein, an instantaneous optimum choice (which may be 
less optimum than the really optimum choice because of 
the shortsighted approach) is employed talcing into 
account the application. Furthermore, complex 
switching is prevented using a threshold value \^ (a 



37 



SC1626 

recent discussion on the effect of switching rules 
upon the stability can be found in D. Liberzon and A. 
Morese, "Basic Problems in stability and Design of 
Switched Systems", IEEE Control Systems Magazine, Oct. 
1999) . 

Although LiapunoV s method can be applied to 
nonlinear dynamics, the execution thereof is 
complicated and it is necessary to perform a computer 
simulation to check whether the algorithm is trapped. 
Pig. 7A is a graph showing the performance index, and 
Fig. 7B is a graph showing the number of local 
interactions. In Pig. 7B showing the number of local 
interactions, open bars, represent the number of 
collisions and solid bars represent the number of role 
replacements. In both Pigs. 7A and 7B, the horizontal 
axis represents the frame number which corresponds to 
the elapsed time. It can be seen from Figs. 7A and 7B 
that the number of collisions and the performance 
index both decrease with increasing frame number and 
with increasing number of role replacements. 

(A-5-3) Re.al-Time Performance 

The algorithm (Figs. 6 and 8) according to the 
present Invention has been evaluated in terras of the 
stability and the calculation load imposed upon a game 
under the conditions of a frame rate of 50 Hz and. a 
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peak number of instructions of the order of GPLOPS/sec 
(Fig* 9), The algorithm has been executed on the 
PlayStation 2 produced by Sony Computer Entertainment, 
Inc. It has been assumed that the CPU time spent for 
5 the processing associated with the dynamic 

characteristics and the coordination is very short (of 
the order 1% of the total CPU time), and the number of 
members whose roles were replaced was limited to K » 6. 
When the number N of members is small, the processing 
10 associated with the coordination and the processing 
associated with the dynamic characteristic need 
ifi similar calculation loads. However, the processing 

hi associated with the coordination becomes dominant when 

t: N is large. 

N 15 The algorithm A2 shown in Fig. 8 is an example of 

Q an algorithm of a version optimized in terms of the 

M block diagonal system. The algorithm A2 shown in Fig. 

8 has better linear characteristics than the algorithm 

Al shown in Fig. 6, because the core matrices S,, A, B, 
20 and 6 in the algorithm A2 increase in size linearly as 
N increases, while they increase in size quadratlcally 
in the algorithm Al. 

The discrete part in the coordination process in 
the algorithm A2 may be treated as an ordinal 
25 assignment problem according to a technique known in 
the art of operational research. For example, when K 



39 



SC1626 

becomes large, a streamlined version of the simplex 
method (linear programming) known as the 
transportation simplex method or a known algorithm 
developed for assignment problems in the art of 
5 operational research may be employed ^ instead of the 
above-described process of replacing roles using a 
permutation table. 

(A- 6) Practical Use of the Algorithm 

In order to employ the theory according to the 
present invention in practical applications, various 
associated techniques including a modeling technique, 
a simulation technique, a content authoring tool, a 
run-time engine, etc., have been developed. The 
developed prototype is based on the run- time codes of 
Mat lab, Maya, and PlayStation 2. 

(A-6-1) Discussion on Monlinearity 

The efficiency of the coordination process depends 
upon the nonlinearity , that is, upon whether the 
dynamic model, which would be built manually in the 
conventional technique, can be well processed using 
linear feedback, because global optimization Is^ 
assured only for linear dynamic characteristics. 

The computerization of such a task is not easy, 
and specialized knowledge about dynamic 
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characteristics is required- Some practical guidelines 
are presented below. 



(A-6-1-1) Neglecting Nonlinearlty 
5 By way of example, an animation of a group 

including 50 members (bees) is described. In this 
example, the bees in the group behave as a group {have 
linear attraction), while they behave repulsively when 
they come very close to each other (that iSi there is 
10 nonlinearlty and the Newton's law of collision holds}. 
One scene of an animation of the group of bees is 
shown in Fig. 10^ 

In Fig. 10, 50 bees gather within a particular 
region below light. Although a dynamic model employed 
15 herein is not a mechanical model employed in many 
other cases, the dynamic model employed allows 
suppression of acceleration of expansion of members. 
Thus, a certain systematic group behavior is described 
by this dynamic model. 
20 The system equation is given as X^^i = AX^ + BU„ + 

(Xn)* Matrices such as those illustrated conceptually 
in Fig. 11 are required for the calculation according 
to the algorithm Al. In Fig. 11, the matrix G is used 
to provide inputs of members and the matrix S is used 
25 to determine the cost (performance index} such as a 
simple distance. In Fig. 11, non-zero elements in the 
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matrices are represented by shaded areas, and zero 
elements are represented by open areas. For a group 
which includes a large number of members, as is the 
case with the present group of bees, the oalculation 
5 load associated with the multivariable feedback 

becomes greater than that of the simulation. This can 
also be seen from comparison between the sizes of 
matrices A and B and the sizes of matrices 6 and S,. 
For the compensation of the group behavior (for a 
10 constant disturbance), a feedforward gain (D. Powell 
and M. Worlcman, "Digital Control of Dynamic Systems", 
Addison Wesley, 1998) is used. First, the optimum 
feedback is designed from A, Q, and R on the basis 
of the LQ synthesis. The effects thereof are then 
15 evaluated by means of a simulation. 
Q As shown in F±g. 7, although the group is 

h I 

collapsed at an initial stage, bees gradually gather 
into a specified form after a layout is applied when 
the frame number » 21. In the present example, the 
20 number of role replacements is limited to 6 for each 
frame. Because the replacement of roles needs a finite 
period of time, a greater number of members of the 
group can be involved in practice. In spite of 
collisions, the group of bees forms a specified shape. 
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(A-6-1-2) Deductive Approach to Preventing 
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Nonlinearity 

A second example Is now shown to discuss a group 
of airplanes having nonlinear characteristics. To 
handle the present problem, linearity is forcedly 
5 introduced by using a Jacobian which linearly 
simulates the dynamic behavior around a certain 
operating point in a state space « Upon the assumption 
that airplanes fly in formation at a substantially 
constant speedy the Jacobian of the system is 
10 extracted: 



.-.(13) 



15 Fig, 12A is a graph illustrating the evaluation 



function (optimal cost} for the present example as a 
funotioa of the frame number. Fig. 12B is a timing 
chart Illustrating the changes in the state space in 
terms of the roles assigned to six airplanes contained 

20 in the airplane formation of the present example. In 
the present example, the control rule is calculated 
upon the assumption that airplanes fly in formation 
and an operating point is obtained near the point at 
which the Jacobian of the system is extracted. The 

25 layout is changed at frame number 100 and also at 
frame number - 200 thereby triggering the replacement 
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of rolds among airplanes * 

As can be seen from Pig. 12, when the nonlinearity 
is not very strong, as is the case in the present 
example « the performance index generally reduces. The 
5 threshold value is reduced to about 30% in order to 
avoid flicker and sporadic replacement of roles. 

(A-*6-l-3) Inevitable Handling of Nonlinearity 

A third example is now presented to discuss a 

10 group consisting of six fishes, having (I) caudal fins 
fc^ providing a propulsive force, (II) a nonlinear 

w combination of state variables in a hydrodynamic 

y system, and (III) a plurality of nonlinearities caused 

X by conditioned responses to avoid collisions. 

15 When fishes of the group move against the flow of 

Q water, an operating point is presented. The Jacobian 

j?; is extracted in the vicinity of the operating point. 

'■li'itr 

^ The caudal fins are moved in response to an impulse 

Q ■ 

train triggered by a negative-phase signal output by a 
20 clock generator. The motion of the caudal fins (P\ P" 
) can be controlled over each time -average driving 
cycle, as pointed out by Nougaret et al (J.I. Nougaret, 
B. Arnaldi, and F. Multon, "Coarse to Pine Design of 
Feedback Controllers for Dynamic Locomotion", Visual 
25 Computer, 4(10), 1997). 

The nonlinearities cause the state-space 
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trajectories of fishes of the group to become 
complicated! as shown in Fig« 13, and the algorithm 
functions in a manner which is very different from the 
predicted manner. 

Pig, 13A illustrates the performance index in the 
present example as a function of the frame number. Fig« 
13B is a timing chart illustrating the changes in the 
state space in terms of the roles assigned to six 
fishes contained in the group in the present example « 
wherein lines represent respective fishes and the 
vertical coordinate represents targets. When there is 
strong nonlihearity as is the case in the present 
example, the performance index decreases in a random 
fashion* For example, replacement of roles is 
triggered by a change in the layout at frame number » 
300. Spontaneous replacement of roles occurs near 
frame number » 200. 

In some oases, the intrinsic dynamic 
characteristics preclude the above-described motion or 
global searching is essentially required. In such a 
case, it often becomes impossible to reach the 
specified layout. In order to adjust the performance 
of the algorithm to avoid the above problem, itr is 
necessary to perform a numerical simulation which 
needs a very large number of calculations. 
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(A-6-2). Limitations and Remaining Problems to be 
Solved 

The algorithm which has been described above with 
reference to specific exeimples of applications has 



47 



SC1626 

been proposed herein in view of thB known techniques 
incapable of efficiently controlling a dynamic group 
including a plurality of members while maintaining the 
trajectories in stable states even in the presence of 
5 an external disturbance. However, if uncompensated 
nonlinearity is carelessly introduced into the system, 
the approach according to the present invention can be 
defeated* From this point of view, an improvement in 
the robustness of the system is required. 
10 For example, in the present invention, collisions 

are avoided in a modest manner by adding a dynamic 
equation which causes interacting members of the group 
J!; to be coupled (temporarily). However, J is evaluated 

f Without taking into account whether the resultant path 

f'^ 15 includes a collision. If a linear repulsive function 
M is used to avoid collisions, an optimum path including 

no collisions may be obtained. However « in this case, 
P a reduction in the performance occurs. A layout may be 

p issued in accordance with a sequence so as to reduce 

20 the possibility of collisions. However, a numerical 
simulation is always required to check the convergence 
and to adjust the threshold parameter 

When convergence is not assured, the implicit 
nonlinear dynamics can make reaching the specified 
25 layout impossible. A possible eventual means is to try 
to handle the trajectories at time intervals using a 
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nonlinear optimization algorithm. However, this 
approach precludes the interactions* Thus, the object 
to b6 achieved next is to enhance the robustness of 
the state feedback by fostering a better understanding 
5 of the hybrid system and the robustness thereof. The 
incorporation of a better collision avoidance rule is 
also required to enhance the robustness of the 
algorithm. 

10 (A- 7) Conclusions 

A technique has been described above for 
automatically incorporating the control rule analyzing 
the behavior thereby controlling the group in a 
coordinated fashion within the framework of optimum 

15 feedback. The dynamic characteristics which need 



j^-b continuous feedback for stabilization have also been 



discussed. However, unfortunately, the approach needs 
a lot of special knowledge about the dynamics « and 
thus this technique is not universal. 

20 ThuSr in order to release game designers from 

complicated and troublesome tasks, it is essentially 
necessary to develop workflows, software tools, and 
components. It is possible to reduce the calculation 
load by processing the complicated dynamic 

25 characteristics in a sequential or parallel fashion 
using the concept of the switched system. Finally, 
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planning and arguments of semantic representations of 
scenes and heuristic approaches will make it possible . 
to improve the role assignment technique, 

(A-8) Off -Line Algorithm 

At the design stage, the coordination method is 
described by parameters using a matrix obtained by two 
off-line algorithms. A first algorithm (Fig. 14) 
calculates the multivariable optimum feedback on the 
basis of eigenveotor escpanslon and solves the ARE . 
(standard algorithm in LQ synthesis) 

The second off-line algorithm (Pig. 15) 
recursively generates Kl permutations and stores them 
in advance* These calculation results are sequentially 
used at run time to evaluate the role replacement. 
When K = 7, the number of entries of the table becomes 
4,320* The permutation algorithm is described in 
Grimaldi, ^'Discrete an Combinational Mathematics " , 
Addison-Wesley, New York, 1999. 

(B) Entertainment Apparatus 

An embodiment of an entertainment apparatus using 
an animation generated by the control method according 
to the present invention is described below. 

(B-1) Functional Block Configuration 
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Fig, 16 Illustrates, in functional block diagram 
form, main parts of the entertainment apparatus of the 
present embodiment . 

In Fig. 16, reference numeral 101 denotes control 
means used by a game player to input various commands . 
Command data input via the control means 10 i is 
transmitted to game executing means 102. 

The geune executing means 102 includes role playing 
game executing means 103 for executing a game in 
accordance with a predetermined scenario, group 
animation control means 104 for producing various 
groups during the game in accordance with a command 
given by the game player or in accordance with the 
scenario and controlling the produced group, and image 
generation means 105. The game executing means 102 
executes the role playing game in accordance with a 
command given by the game player via the control means 
101 and also in accordance with the game program and 
game data stored in storage means 106, Furthermore, 
the game executing means 102 produces an image using 
the image generation means 105 and displays it on 
display means 107. 

Thie game executing means 102 may be realised with 
hardware such as a CPU (central processing unit) and a 
GPU (graphic processing unit). The storage means 106 
may be realized with hardware such as a CD- ROM 



51 



(compact disk-read only memory}, a hard disk^ mdinory, 
or a DVD (digital video disk). 

The display means 107 may be realized with a 
monitor using a CRT (cathode ray tube) or a television 
set. The display means 107 may be combined together 
with or separated from the main unit of the video game 
machine, depending on the type of the game machine, 
that iSt depending upon whether the game machine is 
for home use or for business use* 

Although in the present embodiment , the method of 
controlling a group animation according to the present 
invention is applied to the role playing game, the 
application of the present invention is not limited to 
such an embodiment. The present invention may be 
applied to various games in which a group including a 
plurality of members appears ^ if the dynamic 
characteristics of the respective members can be 
represented in an approximate fashion* For example, 
the present invention may be applied to a war 
simulation game and a race simulation game in which 
groups of human begins and groups of machines appear 
in various forms and move, and a game including 
artificial organisms simulating natural organisms 
which gather into a group and behave as a group. 

(B-2) Hardware Configuration 
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An example of the appearance and an example of the 
general hardware configuration of the entertainment 
apparatus of the present embodiment are described 
below with reference to Pigs, 17 and 18. 

Fig* 17 Illustrates an example of the appearance 
of the entertainment apparatus. The entertainment 
apparatus 1 reads a game program stored on an optical 
disk such as a CD-ROM and executes it in response to 
an operation performed by a game player. 

The main unit 2 of the entertainment apparatus 1 
includes a disk loading part 3 in which an optical 
disk on which a game program is stored is loaded, a 
reset switch 4 for resetting the game, a power switch 
5, a disk control switch 6 for controlling the loading 
of the optical disk, and slots 7A and 7B (two slots in 
this specific example). 

The slots 7A and 7B are used to connect a 
controller 20. The slots 7A and 7B may also be used to 
connect a memory card so that game data is stored to 
or read from it. 

The entertainment apparatus 1 is connected to a 
monitor 30 and loudspeakers 40, although connecting 
parts are not shown in Fig. 17. In the example shown 
in Fig. 17, a television set is used as the monitor 30 
and also as the loudspeakers 40. 

Pig, 18 illustrates the general hardware 
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configuration of the entertainment apparatus 1, As 
shown In Fig. IB, the entertainment apparatus 1 
includes a control system 50 including a CPU 51 and 
associated peripheral devices, a graphic system 60 
5 including a GPU 62 which writes an image into a frame 
buffer 63, a sound system 70 including an SPU (sound 
processing unit) 71 for producing musical sounds or 
sound effects, an optical disk control system 60 for 
controlling the optical disk on which the game program 

10 is stored, and a communication control system 90 for 
controlling the inputting of a signal output by the 
controller 20 in response to a command given by a game 
player and also for controlling the 
inputting/outputting of data to/from a memory card 10 

16 in which settings of the game or the like are stored. 

In Pig. 18, the control system 50 includes the CPU 
51, a peripheral device controller 52 for controlling 
interrupt handling and direct memory access transfer 
operations, a random access memory (RAM) serving as a 

20 main memory 53, and a read only memory (ROM) 54 in 
which is stored an operating system (OS) for 
controlling various parts such as the main memory 53, 
the graphic system 60, and the sound system 70. 
When electric power is turned on, the CPU 51 

25 executes the operating system stored in the ROM 54 so 
as to control the entire entertainment apparatus 1 
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including the graphic system 60, the sound system 70, 
etc. 

After executing the OS, the CPU 51 performs 
initialisation and checks whether various parts work 
correctly- The CPU 51 then controls the optical disk 
control system 80 and executes the game program stored 
on the optical disk. During the execution of the game 
program, in response to a command given by the game 
player, the CPU 51 controls the graphic system eo and 
the sound system 70 so as to display an image and 
generate a sound such as musical sounds or sound 
effects • 

In Fig. 18, the graphic system 60 includes a GTE 
(geometry transfer, engine) 61 for performing a process 
such as a coordinate conversion, a GPU 62 for writing 
an image in accordance with an image write command 
given by the CPU 51, a frame buffer 63 for storing the 
Image written by the GPU 62, and an image decoder 64 
for decoding image data coded in a compressed fashion. 

The GTE 61 has a parallel processing capability 
for performing a plurality of calculations in parallel. 
This allows the GTE 62 to perform, at a high speed, 
various calculations including a coordinate 
transformation, a perspective transformation, and 
lighting calculations in response to calculation 
commands given by the CPU 51. The GPU 62 writes an 
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Image composed of polygons into the frame buffer 63 
according to an image write command given by the CPU 
SI- 

The frame buffer 63 is composed of a dual port RAM, 
5 and is capable of simultaneously reading data to be 
displayed and storing an image written by the GPU 62 
or transferring data from the main memory. The frame 
buffer 63 ;Lncludes a display area for storing video 
data to be displayed on the monitor 30 . The frame 
10 buffer 63 further includes a CLUT area for storing a 
color look-up table (GLUT) which is referred to when 
W the GPU 62 writes a polygon image, and a texture area 

for storing texture data which is inserted « after 
being transformed in terms of coordinates, into 
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15 polygons written by the GPU 62. 
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The image decoder 6.4 decodes, under the control of 
the CPU 51, image data such as still image data or 
moving image data stored in the main memory 53 and 
restores the decoded image data in the main memory 53. 

20 The decoded image data may also be stored by the GPU 
62 into the frame buffer 63 so as to represent a 
background image. 

In Fig. 18, the sound system 70 includes the SPU 
71 for producing sounds such as musical sounds or 

25 sound effects in accordance with a command given by 
the CPU SI, and also includes a sound buffer 72 used 
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by the SPU 71 to store waveform data. The musical 
sounds or sound effects produced by the SPU 71 is 
output to the loudspeakers 40. The SPU 72 further has 
the capability of decoding audio data and also the 
5 capability of reproducing, directly or after 
modulating r waveform data. 

In Fig. 18, the optical disk control system 80 
includes an optical disk drive 81 for reading the game 
program or data stored on the optical disk, a decoder 
10 82 for decoding the game program or the data, and a 
buffer 83 for temporarily storing data read by the 
optical disk drive 81 thereby allowing data to be read 
|ij from the optical disk at a higher speed, 

ass 

% Furthermore, in Fig. 18, the communication control 

15 system 90 includes a communication controller 91 for 
controlling the communication with the CPU 51 via a 
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bus. The communication controller 91 includes a 

controller connecting part 9 for the connection with 

the controller 20 used by a user to input a command 
20 and also includes a memory card insertion part 8 for 

the connection, with the memory card 10 for storing the 

game settings data or the like. 

The controller 20 connected to the controller 

connection part 9 includes a plurality of control keys 
25 used by the game player to input commands. In response 

to a command from the communication controller 91, the 
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controller 20 transmits control key status data. The 
communication controller 91 transfers the control key 
status data received from the controller 20 to the CPU 
51. Thus, the command issued by the game player is 
5 input to the CPU 51, The CPU 51 performs a process 
corresponding to the command given by the game player, 
in aocordance with the game program being executed. 

When it is required to store the settings data or 
the like of the game being executed, the CPU 51 

10 transmits the data to be stored to the communication 
controller 91 which in turn writes the data received 
from the CPU 51 into the memory card 10 loaded in the 
slot of the memory card insertion part 8. 

One example of the entertainment apparatus and one 

15 example of the general hardware configuration thereof 
have been descried above. However, the present 
invention is not limited to the details of those. 
Moreover, the embodiment of the present invention can 
be carried out not only by the entertainment apparatus 

20 but also by a general-purpose information processor* 
For example, when a group animation is displayed 
on the entertainment apparatus using the method of 
controlling the group animation according to the 
present invention, a control key for issuing a group - 

25 level command to create a group or to specify the form 
of the group may be disposed, in addition to the usual 
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control keys* on the controller 20, 

The method of producing a group animation 
according to the present invention and the 
entertainment apparatus using this method have been 
described above. The present invention may also be 
employed to control the motion of a system including a 
plurality of interchangeable members or objects whose 
dynamic characteristics are simulated. 

For example, the present invention may also be 
applied to a method of controlling a system for 
simulating a group including a plurality of 
interchangeable members, the method comprising a 
continuous processing step for determining the states 
of the respective memberd at each time step in 
accordance with predetermined dynamic characteristics 
of the respective members; and a discrete processing 
step for accepting^ from the outside, a command 
specifying the overall state to be achieved for the 
group and assigning roles to the respective members of 
the group in accordance with the overall state 
specified by the command. If new roles are assigned to 
members in the discrete processing step, the dynamic 
states of the respective members are adjusted^ in the 
continuous processing step, in accordance with the 
assigned new roles. 

Furthermore, the present invention may also be 
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applied to a method of, in a simulation of a group 
including N (integer equal to or greater than 2) 
memhers, controlling the movement o£ members of the 
group BO that the members are moved from locations in 
5 a predetermined initial layout In a state space to 
locations in a target layout. This method comprises 
the steps oft 

a) assigning locations in the target layout to the 
respective members lying at locations in the initial 

10 layout so that the members are moved along the 
^ shortest distances to the locations in the target 

layout ; 

b) moving the members in the initial layout in 
accordance with the assignments made in step a); 

F 15 c) calculating the value of a predetermined 

evaluation function associated with the movements, 
accomplished in step b)« of the respective members to 
the assigned locations in the target layout; 

d) selecting K (Integer equal to or smaller than 
20 N) members having the greatest values of the 

evaluation function; and 

e) replacing the assignments of the locations in 
the target layout within Kl combinations of the 
selected K members, 

25 After completion of step e), the method returns to 

step b) so as to perform steps b) to e) repeatedly. 
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